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Abstract 

A  major  factor  that  contributes  to  the  high  rates  of  medical  error  in  the  treatment 
of  patients  with  diabetes  and  other  chronic  diseases  is  the  complexity  of  the  tasks 
that  physicians  must  complete.  SimCare  is  a  model  of  the  clinical  care  setting  for 
patients  with  type  2  diabetes.  The  model  was  designed  to  support  investigation  of 
physician  cognition  and  decisionmaking  activity.  SimCare  is  dynamic  and 
interactive  and  simulates  diabetes  management  in  the  office-practice  setting. 
SimCare  presents  a  series  of  cases  based  on  clinical  situations  representing  task 
features  that  are  thought  to  be  the  source  of  both  realistic  care  decisions  and 
medical  errors.  Once  a  simulated  clinical  case  is  initiated,  physicians  select 
treatment  options  (tenned  “moves”)  from  an  unguided  set  of  choices  similar  to 
those  available  in  routine  office  practice.  The  cumulative  record  of  the  chosen 
treatment  moves  is  available  for  analysis  and  comparison  with  an  expert’s 
sequence  of  moves  for  each  simulated  patient.  SimCare  is  potentially  both  an 
assessment  and  a  teaching  tool  that  enables  the  observation  and  analysis  of 
decisionmaking  in  the  simulated  practice  setting.  This  paper  discusses  the  use  of 
this  tool  to  identify  potential  sources  of  medical  errors  and  guide  customized 
learning  interventions  designed  to  reduce  them. 


Clinical  decisions  in  diabetes  care 

The  clinical  course  of  chronic  illnesses  such  as  diabetes  mellitus  type  2  is 
generally  characterized  by  slowly  changing  states  of  health,  with  inherent 
variability  in  the  rate  of  disease  progression  across  patients.  The  detection  of 
significant  change  in  health  status  of  individual  patients  may  be  masked  by  the 
subtle  progression  of  the  disease  and  its  complications,  and  of  common  comorbid 
conditions.  Physicians  often  fail  to  prescribe  appropriate  evidence-based 
treatment  in  patients  who  have  not  achieved  recommended  clinical  goals  for  a 
variety  of  reasons.1  There  are  several  confounding  variables  that  affect  patients’ 
response  to  treatment.  For  example,  changes  in  socioeconomic  status  or  health 
insurance  coverage  may  alter  patterns  of  care  or  medication  adherence. 

The  cognitive  challenges  to  physicians  caring  for  those  with  a  complex 
chronic  disease  such  as  diabetes  are  many.  Some  of  the  major  cognitive  tasks 
include — 
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•  Selecting  appropriate  evidence-based  clinical  goals  across  multiple 
clinical  domains  (for  example,  glucose,  blood  pressure,  cholesterol). 

•  Initiating  appropriate  therapy. 

•  Titrating  therapy  to  achieve  and  maintain  desired  evidence-based 
goals. 

•  Detecting  and  effectively  managing  comorbid  conditions,  such  as 
depression,  that  may  interfere  with  diabetes  treatment. 

Computer-based  models  and  simulation  methods  have  been  used  to  better 
understand  and  improve  diabetes  care.  Diabetes  Physiolab™  is  a  proprietary 
system  that  models  disease  at  the  level  of  enzymatic  activity.2  Other  efforts  have 
modeled  general  diabetes  physiology  (Sarimner),3  pharmacokinetics,4  specific 
glucose-insulin  interactions  as  educational  simulators  (AIDA,  DIABLOG),5’ 6  as 
well  as  diabetes  decision-support  systems  (DIAS-NIDDM).7  The  Global  Diabetes 
Model,8  a  stochastic  model  of  type  2  diabetes,  has  been  developed  to  predict 
trends  for  diabetic  individuals  or  populations.  A  recent  model — Archimedes9 — 
has  simulated  a  continuous  disease  process  at  the  individual  patient  level. 

SimCare  adds  to  these  efforts  in  a  number  of  respects.  First,  SimCare  was 
designed  to  explore  variability  in  physician  decisionmaking  and  to  map  patterns 
of  chronic  disease  care.  Second,  primary  care  physician  involvement  in  its 
development  assures  that  SimCare  reflects  the  clinical  terrain  and  cognitive 
challenges  associated  with  primary  care  practice.  Third,  SimCare  focuses  on  the 
outpatient  clinical  encounter  as  the  primary  level  of  description  and  interaction. 
SimCare  has  a  broad  therapeutic  scope  that  includes  the  effect  of  oral  diabetes 
medications,  effects  from  referrals  and  advice  to  patient,  in  addition  to  the  effect 
of  prescribed  insulin. 


The  SimCare  simulation 

SimCare  provides  a  dynamic  and  interactive  simulation  environment  for  the 
treatment  of  patients  with  type  2  diabetes.  A  set  of  outpatient  cases  is  presented  in 
sequence.  Each  case  is  managed  through  a  series  of  virtual  patient-physician 
encounters  in  a  primary  care  clinic  setting.  The  physician  is  able  to  perfonn  the 
traditional  subjective-objective-assessment  plan  (SOAP)  approach  to  patient  care. 
This  fonnat  helps  establish  clinical  plausibility  and  provides  a  framework  for 
presenting  data  and  accepting  treatment  moves.  (Each  treatment  action  made  by 
the  subject  physician  through  the  interface  is  a  “move.”) 

The  simulation  is  an  abstraction  of  the  type  2  diabetes  clinical  setting,  created 
for  purposes  of  capturing  physician  treatment  decisions.  The  goal  of  the 
simulation  is  to  engage  physician  decisionmaking  and  solicit  representative 
treatment  moves.  Given  this  focus  on  the  clinical  encounter,  the  simulation  does 
not  incorporate  a  complete  model  of  diabetic  pathophysiology. 

The  simulation  begins  with  the  selection  of  a  patient  case  from  a  library  of 
cases.  Each  case  is  presented  in  the  form  of  a  patient  case  history,  with  initial 
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encounter  data.  The  case  data  is  composed  of  a  set  of  attributes  (variables)  that 
change  as  a  function  of  physician  moves.  In  each  encounter,  the  simulation  moves 
through  a  cycle  composed  of  presentation  of  patient  data,  collection  of  treatment 
moves,  and  generation  of  new  patient  data  in  response  to  moves.  At  the 
conclusion  of  each  encounter,  the  physician  schedules  a  subsequent  visit.  The 
simulation  responds  by  presenting  a  new  encounter  with  updated  patient 
information.  The  update  of  the  patient  information  is  based  on  both  the  physician 
moves  and  on  the  amount  of  elapsed  time.  A  unique  disease  trajectory,  which 
depends  on  the  specific  series  of  moves  made  by  the  physician,  unfolds  for  each 
simulated  patient. 


SimCare  components 

The  SimCare  simulation  has  two  parts:  (1)  a  user  interface  (UI)  that  presents 
the  patient  state  information  and  accepts  physician  moves,  and  (2)  a  patient  model 
(PM)  that  updates  the  patient  state  in  response  to  the  treatment  moves  and  the 
passage  of  time  (Figure  1).  The  UI  provides  the  environment  of  a  clinical 
encounter.  The  PM  generates  information  needed  by  the  physician  to  make 
decisions  with  respect  to  treatment  moves. 

The  user  interface  (UI) 

The  SimCare  UI  presents  patient  data  to  the  physician,  transmits  physician 
moves  to  the  patient  model,  and  preserves  the  clinical  environment  of  continuous 
care  over  the  encounters  during  which  the  patient  is  treated.  There  are  two 
requirements  of  the  UI:  (1)  the  presented  patient  data  must  be  organized  and 


Figure  1.  Physician  interaction  with  SimCare 
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displayed  in  formats  familiar  to  physicians,  and  (2)  the  physician  should  be  able 
to  make  treatment  moves  in  a  familiar  and  acceptable  manner. 

The  UI  allows  the  physician  to  view  patient  information,  including  the 
physical  exam  report,  recent  lab  results,  current  prescriptions,  and  a  patient  self- 
report.  Explicit  physiological  data  (such  as  blood  pressure,  glycated  hemoglobin 
[HgbAlc],  or  a  lipid  panel)  is  provided  in  the  physical  exam  report  and  laboratory 
results.  Implicit  behavioral  (diet,  exercise)  and  psychosocial  information 
(adherence,  depression)  is  provided  through  a  patient  self-report.  The  physician 
may  change  medications,  give  advice  to  the  patient,  order  laboratory  tests,  and 
make  referrals  to  specialists.  At  the  conclusion  of  each  encounter,  the  physician 
can  schedule  a  subsequent  visit. 

A  medication  order  form  consists  of  a  formulary  and  an  insulin  order  matrix 
that  allows  the  physician  to  prescribe  different  types  of  insulin  at  specific  times  of 
the  day.  A  referral  order  form  and  laboratory  test  order  form  are  modeled  on  those 
used  in  a  large  medical  group  practice.  The  physician  is  able  to  give  advice  to  the 
patient  on  diet  and  exercise. 

Treatment  moves  made  by  the  physician  are  collected  by  the  UI  and  sent  to 
the  patient  model,  which  responds  with  an  updated  patient  state.  The  new  patient 
state  infonnation  is  sent  to  the  UI  and  presented  to  the  physician  in  the  next 
encounter  in  the  form  of  reports  comprised  of  (1)  a  subjective  patient  self-report, 
(2)  an  objective  physical  exam  report,  (3)  a  self-monitored  blood  glucose  (SMBG) 
log,  (4)  laboratory  test  results,  and  (5)  referral  response  (if  appropriate). 

The  patient  model  (PM) 

The  PM  is  comprised  of  patient  state  attributes  and  a  set  of  functions  that 
update  values  of  these  attributes  in  response  to  physician  moves.  The  PM  takes 
into  account  the  passage  of  time,  i.e.,  the  patient  state  update  is  contingent  upon 
the  time  elapsed  between  encounters,  in  addition  to  the  physician  moves.  Patient 
state  attributes  are  grouped  into  physiological,  psychosocial,  and  behavioral 
categories.  The  physiological  attributes  specify  the  physiological  condition  of  the 
patient,  while  the  behavioral  attributes  specify  the  current  patient  self-care 
behaviors.  The  psychosocial  attributes  affect  patient  compliance  with 
prescriptions  and  physician  recommendations. 

The  patient  state 

The  set  of  physiological,  behavioral,  and  psychosocial  attributes  that  comprise 

the  patient  state  in  the  SimCare  PM  are  shown  in  Figure  2.  The  physiological 

attributes  change  with  respect  to  time  and  in  response  to  physician  moves.  The 

selection  of  attributes  is  based  on  clinical  treatment  logic,  as  reflected  in  recent 

clinical  guidelines  and  clinical  literature  regarding  type  2  diabetes  mellitus.10' 11 

These  attributes  are  key  features  in  primary  care  management  of  patients  with 

12 

diabetes  and  collectively  address  common  sources  for  physician  errors: 
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Figure  2:  Physiological,  psychosocial,  and  behavioral  attributes  of  the  patient  model 
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•  Insufficient  awareness  and  treatment  of  cardiovascular  risk  factors. 

•  Confounding  effects  of  depression  on  adherence  to  treatment 
regimen.13 

•  Complications  from  drug  therapies.10 


The  adherence  attribute,  which  determines  the  extent  to  which  the  patient 
takes  prescribed  medications  and  follows  physician  recommendations  for  self- 
care,  reproduces  the  effect  of  variation  in  patient  compliance  on  patient  behavior. 
The  effect  of  the  adherence  attribute  is  modeled  by  giving  the  simulated  patient  a 
proportion  (a  value  between  0  and  1)  of  a  given  prescription,  as  specified  by  the 
adherence  value.  For  example,  a  patient  prescribed  Glipizide  (20  mg  q.d.)  who 
has  an  adherence  of  0.5  will  only  take  20*0.5  =  10  mg  q.d.  The  patient  follows 
the  recommended  diet  and  exercise  to  the  extent  dictated  by  the  value  of 
adherence.  The  effect  of  adherence  as  a  moderator  on  all  inputs  to  the  functions  in 
the  PM  is  shown  in  Figure  3.  Depression,  common  among  patients  with 
diabetes,13  is  one  of  the  factors  that  adversely  affect  patient  adherence  to 
medication  and  self-care  behavior  regimens.14  A  patient  self-report  provides  a 
narrative  in  each  encounter  that  describes  self-care  behaviors  and  depression 
symptoms,  if  any.  Patient  adherence  and  depression  are  inferred  from  the 
narrative. 
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Figure  3.  Functions  in  the  patient  model 


Physiological  functions 

Every  physiological  attribute  has  an  associated  physiological  function  that 
updates  its  value.  The  inputs  to  a  function  consist  of  factors  that  affect  the  value 
of  the  attribute,  and  the  output  is  the  new  attribute  value.  For  example,  the  inputs 
to  the  HgbAlc  function  are  current  values  of  diet,  exercise,  and  doses  of 
sulfonylureas,  Metfonnin,  thiazolidionediones  (TZD),  and  insulin: 

New  HgbAlc  =  f  (HgbAlc,  diet,  exercise,  sulfonylureas,  Metformin, 
TZDs,  insulin) 

As  shown  in  Figure  3,  each  function  sums  the  contribution  of  multiple 
physician  moves,  and  time,  to  compute  the  new  value  of  the  attribute.  The 
magnitude  of  the  contribution  of  each  move  is  specified  by  the  magnitude  of  the 
move  (e.g.,  drug  dose),  and  its  timing  (when  it  is  given).  The  fonner  is  computed 
using  a  dose-response  table  (specifying  the  magnitude  of  the  effect),  while  the 
latter  is  computed  using  a  dose-response  schedule  (specifying  the  attenuation  of 
the  medication  effect  over  time). 

Dose-response  table.  The  dose-response  table  computes  the  maximum 
expected  benefit  of  a  given  move,  as  well  as  the  incremental  benefits  of  titrating 
(adjusting)  the  move.  In  the  case  of  medications,  a  dose-response  table  specifies 
the  expected  impact  on  the  attribute  that  the  drug  affects  at  different  dosages. 
There  is  a  dose-response  table  for  each  drug  move  and  the  attribute  that  it  affects, 
which  was  adopted  from  the  Staged  Diabetes  Management  Handbook.1  For 
instance,  as  diet  influences  weight,  blood  pressure,  and  blood  glucose,  the  dose- 
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response  tables  for  diet  are  diet-weight,  diet-blood  pressure,  and  diet-blood 
glucose. 

The  dose-response  table  is  implemented  as  a  sum  that  computes  the  effect  of  a 
given  move.  The  utility  of  the  additive  form  is  that  it  can  be  positive  or  negative, 
depending  upon  the  direction  of  the  dose  change,  and  is  able  to  represent  the 
bidirectional  effects  of  incremental  dose  changes  at  any  dose  level.  The  general 
form  of  the  function  representing  a  dose-response  table  is 

AA ttributeDms  =  Sgn(m0  -  ml )  *  X  a\  *  max(min(Dosei+1 ,  max(m0 ,  m  ))  -  max(Dosej ,  min(m0 ,  m  )),  0) 

i=0 

where  m0=  current  drug  dose,  mi  =  last  drug  dose,  and 
Sgn(x)  =  -1  if  x  is  negative, 

1  if  x  is  positive, 


0  if  x  =  0 


Dose-response  schedule.  The  effect  of  a  move  made  by  a  physician  on 
patient  state  changes  over  time.1  From  the  time  that  a  move  is  made,  the  effect  of 
the  move  increases  gradually  until  its  time  of  peak  effect,  when  the  entire  effect  of 
the  move  is  manifest  on  the  target  patient  variable.  The  proportion  of  the  move’s 
effect  commensurate  with  the  amount  of  time  elapsed  is  computed  as  the  patient’s 
response  to  the  change  in  treatment.  This  is  accomplished  in  the  PM  by  using  a 
time  coefficient  that  specifies  the  proportion  of  the  effect. 

Each  move  available  in  the  simulation  has  an  associated  dose-response 
schedule  that  specifies  the  proportion  of  the  move’s  effect  that  depends  on  the 
time  elapsed  since  the  move  was  made.  Some  moves  reach  their  peak  effect  in  a 
few  days;  others  can  take  several  weeks  before  the  entire  effect  is  manifest.  The 
general  form  of  the  function  that  approximates  the  proportion  of  the  drug  effect  as 
a  result  of  time  attenuation  is 


t„  =  a 5 


-T  \ 


\  +  e  ^ 


where  n  is  the  number  of  days  till  peak  effect,  T  is  actual  time  elapsed,  and  a  and 
[:>  are  coefficients  that  determine  the  desired  shape  of  the  curve.  The  following 
functions  are  used  for  drugs  that  reach  peak  effect  in  14  days  and  90  days, 
respectively: 


( 

-r  A 

-1 

( 

zL  \ 

t  =  2  * 

*14  ^ 

l  +  e  2 

and 

t  =  2  * 

*90  ^ 

l  +  e16 

k 

) 

k 

) 

The  simulated  patient  is  updated  from  one  encounter  to  the  next.  The 
incremental  difference  is  computed  by  subtracting  the  total  effect  of  a  move  on 
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the  current  encounter  from  the  total  effect  until  the  last  encounter.  The  dose- 
response  function  facilitates  this  computation  with  the  functional  form 


f  -(Today— doseDatej )  ^ 

-1 

^  -(LastVisitDate  -  doseDatej )  A 

-1 

1  +  e  13 

- 

1  +  e  p 

\  J 

V  J 

where  Today  is  the  date  of  current  encounter,  LastVisitDate  is  the  date  of  last 
encounter,  and  doseDate;  is  the  date  of  move  for  all  moves  i  =  0..n 

The  incremental  effect  of  a  move  at  a  given  time  is  computed  by  multiplying 
the  output  of  the  dose-response  table  with  the  output  of  the  dose-response 
schedule.  The  clinical  impact  of  various  drugs  on  HgbAlc,  low-density 
lipoproteins  (LDL),  or  blood  pressure  (BP)  is  modeled  using  published 
pharmacokinetic  data.15' 16 

Psychosocial  functions 

The  effect  of  adherence  on  physiological  variables  is  modeled  using 
psychosocial  functions  for  adherence  and  depression.  The  inputs  to  the  PM 
function  that  updates  adherence  are  (1)  current  adherence,  (2)  depression,  (3) 
nurse  educator  referral  (if  any),  and  (4)  dietitian  referral  (if  any).  The  inputs  to  the 
depression  function  are  (1)  current  level  of  depression,  (2)  psychologist  referral  (if 
any),  and  (3)  medications  (i.e.,  Zoloft®’). 

Psychosocial  functions  are  modeled  using  fuzzy  logic,  which  allows 
quantification  of  linguistic  variables.  Psychosocial  attributes  of  stress  and 
depression  are  described  with  verbal  modifiers:  “low,”  “medium,”  and  “high.” 

The  relationships  between  the  psychosocial  attributes  are  described 
linguistically — e.g.,  “If  stress  is  high,  then  blood  pressure  increases”  and  “If 
depression  is  high,  then  adherence  is  low.”  The  four-step  method  described  by 
Kosko  in  1993  was  used  for  computing  adherence  from  depression  and  stress:17 

1.  Fuzzification.  Develop  a  membership  function  to  map  the  verbal 
description  of  variable  (low,  medium,  high)  to  a  numeric  scale. 

2.  Rule  base.  Express  each  relationship  between  an  independent 
psychosocial  variable,  such  as  depression,  and  a  dependent  variable, 
such  as  adherence,  in  a  fuzzy  rule.  For  example,  “IF  High  Depression 
THEN  Eow  Adherence.” 

3.  Inferencing.  Compute  the  fuzzy  value  (set  membership)  of  the 
dependent  variable  by  using  the  value  of  the  independent  variable  and 
the  fuzzy  rules. 

4.  Defuzzification.  Convert  the  fuzzy  value  of  the  dependent  variable  to 
a  specific  numeric  value.  We  use  the  “centroid”  method  of 
defuzzification  in  which  the  center  of  mass  provides  the  crisp  value. 
The  center  of  mass  is  calculated  by  first  treating  each  output  set  as  a 
point  mass,  and  then  calculating  the  center  of  gravity  for  those  point 
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masses  distributed  along  the  “x”  axis.  We  then  find  the  center  of 
gravity  of  the  output  point  masses  using  the  formula 

sum  of  the  moments  sum  of  the  masses. 

This  method  favors  the  rule  with  the  greatest  area  as  contrasted  with 
the  alternative  “height”  method,  which  takes  the  value  of  the  biggest 
contributor. 

Adherence  changes  nonlinearly  in  response  to  the  combined  effects  of  stress 
and  depression.  In  one  case,  a  cycle  in  the  psychosocial  functions  causes 
reinforcing  effects  of  changes  between  depression  and  adherence  (Figure  3) — 
creating  a  “virtuous  circle.”  When  depression  is  treated  with  medication,  it 
improves  adherence,  which  in  turn  increases  the  amount  of  depression  medication 
that  is  taken,  improving  depression  further.  In  another  case,  the  patient  adherence 
could  drop  after  depression  symptoms  have  been  resolved.  The  self-report  patient 
narrative  is  generated  in  the  UI  by  a  look-up  table  that  contains  statement 
modifiers  for  low,  medium,  or  high  levels  of  adherence  and  depression. 


SimCare  implementation 

The  simulation  is  implemented  through  (1)  a  UI  programmed  in  Visual 
Basic™,  (2)  an  Access™  relational  database  (DB),  and  (3)  a  rule-based  engine 
that  embodies  the  PM.  The  DB  stores  the  patient  case  library,  historical  and 
current  patient  state  information,  and  a  record  of  physician  moves.  The  simulation 
begins  with  loading  a  patient  case  from  the  case  library  into  the  UI.  At  this  start 
point,  the  DB  consists  of  only  the  initial  encounter  information  for  each  patient  in 
the  case  library.  The  DB  is  updated  with  subsequent  encounters  as  the  simulation 
progresses.  The  UI  and  rule-based  engine  were  developed  using  Visual  Basic  to 
allow  rapid  initial  prototyping  through  an  object-oriented  interface,  and  ease  of 
interfacing  with  the  Access  database  application. 


Validation  and  verification 

Validation  has  been  defined  by  Law  as  “the  process  of  detennining  whether  a 
simulation  model  is  an  accurate  representation  of  the  system,  for  the  particular 
objectives  of  the  study.”  SimCare’s  primary  objective  was  to  represent  a  virtual 
clinical  encounter  for  patients  with  type  2  diabetes  and  to  achieve  sufficient 
clinical  plausibility  that  representative  patient  management  actions  would  be 
elicited. 

In  the  SimCare  project,  Sargent’s  definition  of  model  verification  is  used: 
“ensuring  that  the  computer  program  of  the  computerized  model  and  its 
implementation  are  correct.”19  Software  constructs  such  as  object-oriented 
programming  and  program  modularity  were  used  to  reduce  implementation  error. 
Both  static  “structured  walk-throughs”  and  dynamic  testing  were  perfonned  as  a 
part  of  the  model  verification  process.  Structured  walk-throughs  were  performed 
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with  an  expert  physician  panel  at  two  levels — with  the  conceptual  model  to 
validate  the  simulation’s  representation  (described  below),  and  with  the 
programmed  simulation  to  verify  the  computer  model  implementation. 

Conceptual,  operational,  and  face  validity 

Conceptual  validity  addresses  the  question  of  how  well  the  components  of  the 
model  and  their  interactions  represent  the  phenomenon  under  investigation — in 
this  case,  the  clinical  encounter.  The  conceptual  model  of  the  clinical  encounter 
and  the  key  patient  variables  in  the  treatment  of  type  2  diabetes,  in  the  SimCare 
model,  are  based  on  clinical  treatment  guidelines  (Institute  for  Clinical  Systems 
Integration  [ICSI]),  the  Staged  Diabetes  Management,  and  the  American  Diabetes 
Association  (ADA).1, 10, 11  The  physiological  responses  in  the  PM  are  based  on 
clinical  literature15  and  the  documented  effect  of  psychosocial  factors  and  their 
impact  on  treatment.20' 21  A  panel  of  expert  physicians  stepped  through  the  process 
of  managing  an  encounter  in  the  conceptual  model  description  to  determine  if  the 
representation  of  the  clinical  encounter  was  appropriate  for  SimCare’s  purpose. 
The  SimCare  UI  design  was  based  on  common  formats  of  receiving  information 
and  taking  action  familiar  to  the  physicians  (medication  order  form,  laboratory 
test  order  form,  referral  order  fonn,  physical  exam  report,  etc.). 

Operational  validity  detennines  if  the  model’s  output  “has  the  accuracy 
required  for  the  model’s  intended  purpose  over  the  domain  of  its  intended 
applicability.”19  Model  responses  were  studied  to  determine  if  they  were 

•  Within  valid  clinical  bounds. 

•  Robust  to  extreme  or  multiple  inputs. 

•  Sensitive  to  small  changes  in  the  input. 

These  questions  were  addressed  using  “extreme  condition  tests,”  in  which  the 
model  output  was  examined  for  extreme  and  unlikely  combinations  of  values.  In 
addition,  “degenerate  tests,”  which  involve  the  degeneracy  of  the  model’s 
behavior,  were  performed.  The  tests  conducted  on  SimCare  that  involved 
changing  the  values  of  the  model  inputs  in  small  and  large  increments  to 
determine  the  effect  on  outputs  are  referred  to  as  “parameter  variability-sensitivity 
analysis.”  The  PM  response  to  these  series  of  experiments  were  graphed  and 
reviewed  by  the  panel  of  consulting  physicians. 

Face  validity,  defined  by  Sargent  as  “asking  people  knowledgeable  about  the 
system  whether  the  model  and/or  its  behavior  are  reasonable,”19  was  deemed 
sufficient  for  the  purpose  of  generating  plausible  patient  responses. 

SimCare  experimental  data 

SimCare  was  required  to  fulfill  the  primary  criterion:  “Can  it  be  used  by 
physicians  to  make  decisions  on  patients  who  are  portrayed  in  clinical  contexts?” 
The  patient  response  had  to  be  plausible  not  just  for  a  particular  physician  move, 
but  for  a  series  of  moves.  Table  1  shows  a  summary  of  experimental  data  for  one 
subject,  and  Figure  4  is  a  graph  of  the  subject’s  moves  that  is  available  as  a 
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followup  to  the  experiment.  Table  1  has  three  sections:  (1)  lab  test  orders  and 
referrals,  (2)  medication  moves,  and  (3)  the  patient’s  state.  Each  column  shows  a 
physician-patient  encounter.  The  table  entries  show  the  time  stamp  in  parentheses 
for  when  a  move  was  made.  For  example,  “*(327)”  in  the  first  encounter  (Lab 
Orders  section/TSH  Order  row)  indicates  that  a  measurement  of  thyroid 


Table  1.  Experimental  data:  decision  graph  of  a  physician’s  treatment  of  a  case 

Decision  Graph  for  Physician  5X 


Ptntld  16 


Visit  Number 

0  (Initial  data) 

1 

2 

3 

4 

Encounter  Date 

01/01/2001 

01/15/2001 

02/12/2001 

03/12/2001 

Visit  Type 

Visit 

Visit 

Visit 

Phone 

Lab  Orders 

HgbAlc 

v(94) 

v(735) 

v(885) 

SMBG 

v(481  )*(529) 

v(600) 

v(1000) 

v(1079) 

Creatinine 

v(98)*(31 1 ) 

v(740) 

v(891) 

Lipids 

v(95) 

v(737) 

ALT 

*(309) 

*(327) 

UMACR 

‘(332) 

CPK 

*(314) 

GL 

*(337) 

Referrals 

Psych, DE, 
Opth, Diet 

Endo 

Meds 

Aspirin 

325q.d.(228) 

325q.d. 

325q.d. 

Atorvastatin 

20q.d. 

40q.d.(217) 

40q.d. 

40q.d.(980) 

Hydrochlor 

25q.d. 

0q.d.(291) 

Lisinopril 

10q.d.(299) 

lOq.d. 

lOq.d. 

Metformin 

500b. i.d. 

500b. i.d. 

0b. i.d. (771 ) 

Rosiglitazone 

4q.d.(956) 

Zoloft 

50q.d. 

50q.d. 

Insulin 

75R  60N 

75R 

65N(485) 

81 R 

75N(681 ) 

81 R 

75N(1015) 

81 R 

75N(1090) 

Patient  State 

Ale 

10.5 

10.5 

9.9 

8.3 

7.4 

Avg.  SMBG 

267 

261 

229 

165 

146 

Adherence 

.5 

.5 

1.0 

1.0 

1.0 

LDL 

157 

157 

139 

120 

117 

HDL 

28 

27 

30 

32 

33 

Triglycerides 

380 

380 

332 

263 

252 

Systolic 

150 

153 

138 

131 

131 

Creatinine 

1.5 

1.5 

1.8 

1.9 

1.9 

Depression 

60 

60 

30 

0 

0 

Legend:  *  =  Lab  Order,  v  =  View  Lab  Values,  DE  =  Diabetes  Educator,  Diet  =  Dietician,  Opth  = 
Ophthalmologist,  Pod  =  Podiatrist,  Psych  =  Psychiatrist.  The  number  in  parenthesis  indicates  the 
time  stamp  of  the  move. 

(All  dosages  are  qd  unless  otherwise  noted.) 
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stimulating  hormone  was  made  327  seconds  into  the  case.  The  physician  subject 
is  able  to  view  only  those  patient  state  variables  for  which  a  test  was  ordered. 
Depression  and  adherence  are  conveyed  through  the  patient  response  and  self- 
report  narrative. 

Figure  4  shows  a  graph  of  the  patient  HgbAlc  response  to  physician  moves. 
Similar  graphs  are  available  for  lipids  and  blood  pressure.  In  the  project  for  which 
the  simulation  was  developed,  40  physicians  treated  a  set  of  simulated  patients 
using  the  SimCare  model. 


Summary  and  conclusion 

SimCare  has  several  limitations.  First,  the  SimCare  UI  departs  from  the  actual 
encounter  environment,  as  it  removes  some  of  the  barriers  faced  in  actual 
encounters  and  provides  an  ease  of  performing  moves  that  is  not  found  in  actual 
care.  Second,  a  computer  simulation  is  unable  to  replicate  the  wealth  of 
information  that  the  physician  gathers  from  a  conversation  and  face-to-face 
meetings  with  a  patient.  In  the  SimCare  simulation,  the  physician  is  limited  to 
behavioral  recommendations  to  the  patient  and  a  patient  self-report.  Third, 
SimCare’s  abstracted  encounter  minimizes  the  reality  of  potential  psychosocial 
and  economic  barriers  to  care  that  patients  face  in  reality. 

Despite  these  limitations,  the  SimCare  simulation  has  a  variety  of  potential 
applications.  First,  the  information  obtained  can  be  a  powerful  tool  in  the  effort  to 

Figure  4.  Experimental  data:  HgbAlc  trace 


HgbAlc  process  trace  tor  Tot 


M -  2 


1/1/2001  1/1/2001  1/1S/20O1  2/12/2001  3/12/2001 

Encounter 

Metformin  1000 
Insuto  140 

_  Insuwi  156 

Rosiglilazone  4 

Dose  and  Duration  of  Medication 


190 


Modeling  Decisionmaking  in  Diabetes  Care 


identify  appropriate  moves,  as  well  as  errors,  in  the  care  of  adults  with  diabetes.  It 
is  possible  to  map  a  given  physician’s  decision  processes  with  respect  to  diabetes 
care,  and  identify  both  specific  strengths  and  opportunities  for  improvement  in  the 
core  cognitive  tasks  that  underlie  decisions  regarding  chronic  disease  care. 

Examples  of  problems  uncovered  by  analysis  of  the  SimCare  case 
management  data  include  failure  to  recognize  comorbid  depression  (as  indicated 
by  lack  of  treating  depression  in  instances  where  the  patient  is  not  responsive  to 
treatment  and  self-reports  depressive  symptoms);  inability  to  initiate  insulin  when 
necessary;  failure  to  titrate  insulin  as  appropriate;  failure  to  recognize  evidence- 
based  goals  for  HgbAlc,  BP,  or  lipid  control;  and  lack  of  familiarity  with 
common  contraindications  to  the  use  of  medications  (such  as  Metformin). 

Physician  performance  on  the  SimCare  cases  can  also  be  used  to  guide 
learning  interventions  tailored  to  a  given  physician’s  patterns  of  care.  Thus,  if  one 
physician  demonstrates  underuse  of  therapy  such  as  insulin,  teaching  cases  that 
emphasize  key  aspects  of  insulin  treatment  may  be  provided  as  a  learning 
intervention  for  that  physician.  Likewise,  physicians  who  habitually  miss  the 
diagnosis  of  comorbid  depression  can  receive  learning  interventions  that  help 
develop  that  skill. 

With  additional  development,  SimCare  cases  could  be  used  proactively  with 
physicians  early  in  their  professional  careers,  to  guide  the  development  of  core 
competencies  in  chronic  disease  care.  The  cases  were  given,  as  a  pilot  study,  to  a 
series  of  family  practice  residents  and  faculty  who  were  able  to  recognize  the 
value  of  the  simulated  case  management  as  a  powerful  clinical  teaching  tool.  The 
relevance  and  potential  application  of  such  learning  tools  may  increase  due  to 
statutory  limits  imposed  on  residency  training  hours  and  the  need  to  supplement 
“real”  patient  care  experiences  with  high-volume,  “simulated”  cases  with 
embedded  teaching  points. 

Finally,  the  model  now  includes  measures  of  resource  use  linked  to  visits, 
prescriptions,  tests,  and  referrals.  These  measures  enable  physicians  to  receive 
feedback  on  the  ratio  of  clinical  benefit  to  resource  use  engendered  for  specific 
patterns  of  diabetes  care.  This  may  lead  to  recognition  by  individual  physicians  of 
inefficient  patterns  of  resource  use  in  diabetes  care.  A  further  extension  would  be 
to  use  the  model  to  simulate  the  costs  of  various  diabetes  care  protocols  across 
populations  of  diabetes  patients. 
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